Component-Based Construction of a Science Learning Space

We present a vision for learning environments, called Science Learning Spaces, that are rich in engaging content and activities, provide constructive experiences in scientific process skills, and are as instructionally effective as a personal tutor. A Science Learning Space combines three independent software systems: 1) simulations in which experiments are run and data is collected, 2) representation construction tools in which data is analyzed and conceptual models are expressed and evaluated, and 3) tutor agents that provide just-in-time assistance for acquiring higher order skills like experimental strategy, representational tool choice, conjecturing, and argument. Achieving the Science Learning Space vision will require collaborative efforts facilitated by a component-based software architecture. We have created a feasibility demonstration that serves as an example and a call for further work toward achieving this vision. In our demonstration, we combined 1) the Active Illustrations simulation environment, 2) the Belvedere evidence mapping environment, and 3) a model-tracing Experimentation Tutor Agent. We illustrate student interaction in this Science Learning Space and discuss the requirements, advantages, and challenges in creating one. Koedinger, K.R., Suthers, D.D., Forbus, K.D. The Science Learning Space Vision Imagine an Internet filled with possibility for student discovery. A vast array of simulations is available to explore any scientific field you desire. Easy-to-use data representation and visualization tools are at your fingertips. As you work, intelligent tutor agents are watching silently in the background, available at any time to assist you as you engage in scientific inquiry practices: experimentation, analysis, discovery, argumentation. This is our vision for Science Learning Spaces. Table 1 summarizes how this vision contrasts with typical classroom experience. Table 1. What Science Learning Spaces Have to Offer Typical Science Class Science Learning Space Vision Content Lectures, fixed topics, fixed pace, focus on facts Vast options, student choice and pace, learning both concepts and process skills Activity Inquiry process hampered by mundane procedure & long waits Simulations speed time leave technique lessons for later Tools Paper and pencil Data representation & model construction tools Assistance Limited, 1 teacher for 30 students Automated 1:1 assistance of tutor agents Assessment Large grain, limited assessmentinstruction continuity Tutor agents monitor student development at action level Unlike the traditional science classroom of lectures, fixed topics and a fixed pace, in a Science Learning Space the students choose areas of interest, work at their own pace, and engage in learning by doing. Unhampered by “test tubes and time,” they work with simulations that simplify, accelerate, and make possible otherwise mundane, time-intensive, dangerous, or even impossible experiments and investigations. In addition to recording data and ideas on paper, students use sophisticated computer-based tools for data representation and model construction. Rather than competing with 30 other classmates for help from one teacher, each student can always get assistance from one or more computerized tutor agents in addition to occasional help from the teacher. In effect, the classroom has a teacher’s aid for every student. Finally, when it Component-Based Construction of a Science Learning Space comes to assessment, the teacher can supplement occasional paper-based assessments with the detailed, continuous assessment provided by tutor agents. Unlike paper assessments, tutor agents capture students’ success or difficulty with specific concepts and skills, and do so automatically and in the context of complex activities. This powerful combination of content, activity, tools, assistance, and assessment is achieved within a Science Learning Space by coordinating software components of three types: 1) simulations in which experiments are run and data is collected, 2) representation construction tools in which data is analyzed and conceptual models are expressed and evaluated, and 3) tutor agents that provide just-in-time assistance in acquiring higher order skills like experimental strategy, representational tool choice, conjecturing, and argument. Although the full Science Learning Space vision is currently out of reach, we have created a demonstration of its feasibility. This demonstration serves as a model for future work and a call for further community efforts directed toward achieving this vision. The Need for Collaborative Component-Based Development Research in intelligent learning environments typically involves designing and implementing an entire system from scratch. Time and resources spent on software engineering is taken away from the education and research the software is designed to support. Today the typical solution in research labs and industry is to work within the context of an in-house software investment, evolving each new system from previous work. This makes replication and sharing more difficult and can lead to maintenance and deployment difficulties as restrictive platform requirements accumulate over time. Koedinger, K.R., Suthers, D.D., Forbus, K.D. This situation is growing intolerable, and so recently there has been a surge of interest in architectures and frameworks for interoperable and component-based systems for education [Ritter & Koedinger, 1997; Roschelle & Kaput, 1995; Roschelle et al. 1998; Suthers & Jones, 1997]. This has led to a number of successful workshops on the topic,1 several standards efforts specifically targeted to advanced educational technologies,2 and new repositories for educational object components.3 These efforts gain leverage from the rise of interactive Web technology and its associated emphasis on standards-based interoperability. Emerging solutions for componentbased systems include development frameworks,4 shared communication protocols,5 markup languages,6 and metadata formats.7 Although the component-based solutions developed to date are useful, they are inadequate for those building component-based intelligent learning environments in which the components must respond to the meaning of the content as well as its form and presentation. We see the development of techniques for sharing semantics across components and applications to be a critical research direction for the field. Recently we conducted a demonstration of the feasibility of integrating three different, independently developed components. Two of the components were complete intelligent learning environments in their own right. Active Illustrations [Forbus, 1997] enable learners to experiment with simulations of scientific phenomena, and to receive explanations about the 1 E.g., “Architectures and Methods for Designing Cost-Effective and Reusable ITSs” at ITS’96 (http://advlearn.lrdc.pitt.edu/itsarch/), and “Issues in Achieving Cost-Effective and Reusable Intelligent Learning Environments” at AI-ED’97 (http://www.fu.is.saga-u.ac.jp/aied/workshop2.html). 2 E.g., ARIADNE's Educational Metadata Recommendation (http://ariadne.unil.ch/metadata.htm), the IEEE 1484 Learning Technology Standards Committee (http://www.manta.ieee.org/p1484/n2index.htm), and Educom’s IMS (http://www.imsproject.org/). 3 E.g., the Educational Object Economy (http://www.eoe.org/) and Gamelan (http://www.developer.com/directories/pages/dir.java.educational.html). 4 E.g., Java Beans (http://java.sun.com/docs/books/tutorial/javabeans/index.html). 5 E.g., CORBA (http://www.omg.org/news/begin.htm). 6 E.g., XML (http://www.w3.org/TR/PR-xml.html). 7 E.g., the IMS/LTSC work mentioned in footnote 2. Component-Based Construction of a Science Learning Space causal influences behind the results [Forbus & Falkenhainer 1990; 1995]. Belvedere [Suthers & Jones, 1997; Suthers et al., 1997] provides learners with an “evidence mapping” facility for recording relationships between statements labeled as “hypotheses” and “data”. A Scientific Argumentation Advisor [Paolucci et al., 1996] guides students to seek empirical support, consider alternate hypotheses, and avoid confirmation biases, among other things. The third component was an instance of a model-tracing Tutor Agent [Ritter & Koedinger, 1997] that contains a cognitive model of general experimentation and argumentation process skills. This “Experimentation Tutor Agent” dynamically assesses student performance and is available to provide students with just-in-time feedback and context-sensitive advice. Our Learning Space Demonstration took place in the context of meetings of DARPA’s Computer Aided Education and Training Initiative program contractors. Using a MOO as a communication infrastructure [O’Day, Bobrow, Bobrow, Shirley, Hughes, Walters, 1998], we demonstrated a scenario in which a student poses a hypothesis in the Belvedere evidence-mapping environment, uses the simulation to test that hypothesis in the Active Illustration environment and sends the results back to Belvedere for integration in the evidence map. Throughout this activity the Experimentation Tutor Agent was monitoring student performance and was available to provide assistance. From this experience we abstracted the notion of a Science Learning Space. In our demonstration, the Space was filled with Active Illustrations as the simulation component, Belvedere as a representation construction tool, and the Experimentation Tutor Agent and Argumentation Advisor in tutor roles. In this paper we discuss how interoperability of these components was achieved through the use of Translator components that enable communication between existing functional components with little or no modification to them. We begin by Koedinger, K.R., Suthers, D.D., Forbus, K.D. examining the constraints that developing intelligent learning environments impose on the nature and types of components and their interactions, focusing on the importance of semantic interoperability. We then describe the demonstration configuration in detail, showing how it exploits a limited form of semantic interoperability. Finally, we reflect on the requirements, advantages, and future directions in creating Science Learning Spaces. COMPONENTS FOR INTELLIGENT LEARNING ENVIRONMENTS Component-based development has a number of purported economic and engineering benefits. Component-based systems are considered to be more economical to build because prior components can be re-used, saving time for new research and development efforts. They are easier to maintain due to their modular design and easier to extend because the underlying frameworks that make component-based development possible in the first place also make it easier to add new components. We can also expect better quality systems as developers can focus their efforts on their specialty, whether in simulation, tool, or tutor development. However, there is a deeper reason why we believe component-based educational software is important: It will enable us to construct, by composition, the multiple functionalities needed for a pedagogically complete learning environment. Various genres of computer-based learning environments have had their advocates. Each provides a valuable form of support for learning, but are insufficient in themselves. Yet today, the high development costs associated with building each type of environment leads to the deployment of systems with only a small subset of desirable functionality. For example, microworlds and simulations enable students to directly experience the behavior of dynamic systems and in some cases to change that behavior, experimenting with alternate models. These environments are consistent with the notion that deeper learning takes Component-Based Construction of a Science Learning Space place when learners construct knowledge through experience8. However, simulations lack guidance: Taken alone, they provide no tools for the articulation and reflection on this knowledge and no learning agenda or intelligent assistance. On the other hand, intelligent tutoring systems provide substantial guidance in the form of a learning agenda, a model of desired thinking and performance, and intelligence assistance. This form of guidance is particularly important in domains where the target knowledge is not an easy induction from interactions with the artifact or system of interest. In such domains, intelligent tutors can lead to dramatic “one sigma” increases in student achievement [e.g., Koedinger, Anderson, Hadley, & Mark, 1997]. However, while intelligent tutoring systems are sometimes designed with sophisticated and novel interfaces [e.g., Koedinger, 1991; Reiser et al., 1991], such interfaces are not the core of the approach. Tutor interfaces are not typically designed to afford and support desired student thinking processes like data manipulation, pattern search, or the expression and testing of new knowledge. A third type of educational component, representational tools , fill this need. Representational tools range from basic office software such as spreadsheets, graphers, and outliners to “knowledge mapping” software and other specialized tools designed based on cognitive analyses to address particular learning objectives [Koedinger, 1991; Reiser et al., 1991; Reusser, 1993]. Properly designed, representational tools can function as cognitive tools that lead learners into knowledge-building interactions [Collins & Ferguson, 1993; Goldenberg, 1995; Kaput, 1995], and guide collaborative as well as individual learning interactions [Roschelle, 1996]. As learner-constructed external representations become part of the collaborators’ shared 8 For a review of research on the features and benefits of simulations see de Jong & van Joolingen (1998). Koedinger, K.R., Suthers, D.D., Forbus, K.D. context, the distinctions and relationships that are made salient by these representations may influence their interactions in ways that influence learning outcomes [Suthers, 1999]. Yet representational tools provide only a subtle kind of guidance. As with simulations and microworlds, direct tutoring interventions are sometimes needed as well. Fortunately there is a double-synergy: Inspection of learners’ representations and simulation actions can provide a tutor with valuable information about what students are thinking and thus what kind of guidance is needed. We believe that the potential to routinely synthesize new intelligent learning environments from off-the-shelf components that combine multiple functionalities is sufficient justification for moving to a component-based development approach. The potential advantages of component-based systems must, of course, be weighed against their costs. Creating composable software components requires exposing enough of their internal representations, through carefully designed protocols, so that effective communication is possible. Doing this in ways that minimize communication overhead while maximizing reuse is a subtle design problem that can require substantial extra work. Communication between multiple development teams in different locations also provides a substantial challenge. A FEASIBILITY DEMONSTRATION OF A SCIENCE LEARNING SPACE In this section we describe the Science Learning Space demonstration that we undertook. We begin with the learning activity that motivates our particular combination of tools; then we describe the underlying architecture and step through an example interaction scenario. The Learning Activity: Scientific Inquiry There is no point in combining components unless the learner benefits. In particular, the functionality provided by each component must contribute to the facilitation of effective learning Component-Based Construction of a Science Learning Space interactions in some way. Consider scientific inquiry. Students have difficulty with the basic distinction between empirical observations and theoretical claims and particularly in coordinating the two (Kuhn, Amsel, O’Loughlin, 1988; Ranney, et al. 1994). They need to learn that theories are posed to explain and predict occurrences and that theories are evaluated with respect to how consistent they are with all of the relevant observed data. They need to seek relevant evidence, both confirming and disconfirming, perform observations, and conduct experiments to test hypotheses or to resolve theoretical arguments between hypotheses. Experimentation requires certain process skills, such as the strategy of varying one feature at a time. Evaluation of the results of experiments requires scientific argumentation skills. Thus, this is a learning problem that could benefit from (1) experimentation in simulation environments, aided by coaching based on a process model of effective experimentation; and (2) articulation of and reflection upon one’s analysis of the relationships between hypotheses and evidence, aided by coaching based on principles of scientific argumentation. The Implementation Architecture We describe the abstract implementation architecture (see Figure 1) behind our demonstration as one illustration of how several technologies enable the construction of component-based systems. Our collaboration began with a Science Learning Space demonstration involving an Experimentation Tutor Agent and Active Illustration [Forbus, 1997] communicating through a Lambda-MOO derivative using the "MOO Communications Protocol" [O’Day, et al., 1998]. Forbus had already made use of the MOO for communication between the Active Illustration simulation engine and a simulation user interface (upper right of Figure 1). A MOO was chosen as the infrastructure because its notion of persistent objects and multi-user design made it easy for participants in experiments (both human and software) to be in a shared environment despite Koedinger, K.R., Suthers, D.D., Forbus, K.D. being on different machines, often in different parts of the country. The open, ASCII-based MOO Communications Protocol made it easy to add a Tutor Agent to monitor student performance as the basis for providing context-sensitive assistance. Koedinger used the plug-in tutor agent architecture [Ritter & Koedinger, 1997] which employs a simple Translator component (box #1, lower right of Figure 1) to manage the communication between tools and tutor agents. The Translator watched for messages between the Simulation Interface and the Active Illustration server, extracted messages indicating relevant student actions, and translated these student actions into the “selection-action-input” form appropriate for semantic processing by the Tutor Agent’s “model-tracing” engine. Model-tracing is a plan recognition technique used in Cognitive Tutors to parse student action sequences in terms of a goal-based production rule sequence that could have generated such actions [Anderson, Corbett, Koedinger, & Pelletier, 1995]. Figure 1. Communication architecture used in the demonstration. M O O Belvedere Interface Experiment Tutor Agent Simulation Interface Active Illustration CLIENT Messages & Skillmeter lvedere dvisor 1 3 2 Component-Based Construction of a Science Learning Space In the first of two Science Learning Space demonstrations we created, the science domain was evaporation processes, a typical middle school science topic. To demonstrate the domain generality of Active Illustrations and the Experimentation Tutor Agent, we changed the science domain in our second Science Learning Space demonstration to atmospheric phenomena, of particular interest because of its relationship to global warming. More significantly, in this second demonstration we added a Representation Construction tool, the Belvedere system, and a second tutor agent, the Argumentation Advisor (see the left side of Figure 1). Belvedere itself provides a communication architecture, described in Suthers & Jones [1997], which is shown in Figure 1 as box #2, between the Belvedere Advisor and Interface. Integration of the Belvedere subsystem into the MOO required the addition of one translator component (box #3 in the figure). No modification to Belvedere itself was required. The translator watched the MOO for Hypothesis and Simulation Run objects sent by the Simulation Interface. When seen, these were converted to Belvedere Hypothesis and Data objects, and sent into Belvedere for consideration. Koedinger, K.R., Suthers, D.D., Forbus, K.D. Figure 2. Opening situation: An initial hypothesis is posed in Belvedere.